Catheter segmentation in X-ray fluoroscopy using synthetic data and transfer learning with light U-nets

Background and objectivesAutomated segmentation and tracking of surgical instruments and catheters under X-ray fluoroscopy hold the potential for enhanced image guidance in catheter-based endovascular procedures. This article presents a novel method for real-time segmentation of catheters and guidewires in 2d X-ray images. We employ Convolutional Neural Networks (CNNs) and propose a transfer learning approach, using synthetic fluoroscopic images, to develop a lightweight version of the U-Net architecture. Our strategy, requiring a small amount of manually annotated data, streamlines the training process and results in a U-Net model, which achieves comparable performance to the state-of-the-art segmentation, with a decreased number of trainable parameters. MethodsThe proposed transfer learning approach exploits high-fidelity synthetic images generated from real fluroscopic backgrounds. We implement a two-stage process, initial end-to-end training and fine-tuning, to develop two versions of our model, using synthetic and phantom fluoroscopic images independently. A small number of manually annotated in-vivo images is employed to fine-tune the deepest 7 layers of the U-Net architecture, producing a network specialized for pixel-wise catheter/guidewire segmentation. The network takes as input a single grayscale image and outputs the segmentation result as a binary mask against the background. ResultsEvaluation is carried out with images from in-vivo fluoroscopic video sequences from six endovascular procedures, with different surgical setups. We validate the effectiveness of developing the U-Net models using synthetic data, in tests where fine-tuning and testing in-vivo takes place both by dividing data from all procedures into independent fine-tuning/testing subsets as well as by using different in-vivo sequences. Accurate catheter/guidewire segmentation (average Dice coefficient of ~ 0.55, ~ 0.26 and ~ 0.17) is obtained with both U-Net models. Compared to the state-of-the-art CNN models, the proposed U-Net achieves comparable performance ( ± 5% average Dice coefficients) in terms of segmentation accuracy, while yielding a 84% reduction of the testing time. This adds flexibility for real-time operation and makes our network adaptable to increased input resolution. ConclusionsThis work presents a new approach in the development of CNN models for pixel-wise segmentation of surgical catheters in X-ray fluoroscopy, exploiting synthetic images and transfer learning. Our methodology reduces the need for manually annotating large volumes of data for training. This represents an important advantage, given that manual pixel-wise annotations is a key bottleneck in developing CNN segmentation models. Combined with a simplified U-Net model, our work yields significant advantages compared to current state-of-the-art solutions

Linear-logarithmic CMOS pixel with tunable dynamic range

Abstract—A CMOS pixel with linear–logarithmic response and programmable dynamic range (DR), based on a tunable transition point, has purposely been designed for endoscopic applications. A theoretical model of the pixel was developed and validated. A chip with a 100 × 100 pixel array and a 12-b digital output was fabricated in a 0.35-μm technology and was fully tested, thus demonstrating state-of-the-art performance in terms of DR and noise. Intraframe DR proved to be extendable to more than 110 dB through a logarithmic compression of the signal in the light ir-radiation power density (LIPD) range. The measured temporal noise (pixel noise) was less than 0.22 % over the full range. The architecture presented limited fixed pattern noise (FPN) due to the scheme adopted, which allowed its correction over the full signal range: FPN was 0.83 % (1.37%) in the linear (logarithmic) region. Although the test chip was designed mainly for endoscopic applications, the technology may also be applied to other fields, e.g., robotics, security and industrial automation, whenever high DR is a crucial feature. Index Terms—CMOS imager, endoscopy, logarithmic response, pixel. I

Urinary Bladder Volume Reconstruction Based on Bioimpedance Measurements: Ex Vivo and In Vivo Validation Through Implanted Patch and Needle Electrodes

Restoring bladder sensation in patients with bladder dysfunctions by performing urinary volume monitoring is an ambitious goal. The bioimpedance technique has shown promising results in wearable solutions but long-term validation and implantable systems are not available, yet. In this work, we propose to implant commercial bioimpedance sensors on bladder walls to perform bladder volume estimation. Two commercial sensor types (Ag/AgCl patch and needle electrodes) were selected to this purpose. Injected current frequency of 1.337 MHz and electrodes pair on the same face of the bladder allowed to correlate the changes in impedance with increasing volumes. Two volume reconstruction algorithms have been proposed, based on the direct correlation between bioimpedance readings and bladder volume (Algorithm A) or bioimpedance readings and inter-electrode distance (Algorithm B, bladder shape approximated to a sphere). For both algorithms, a better fit with a second-degree fitting polynomial was obtained. Algorithm A obtained lower estimation errors with an average of 20.35% and 21.98% (volumes greater than 150 ml) for patch and needle electrodes, respectively. The variations in ions concentration led to a slight deterioration of volume estimation, however the presence of tissues surrounding the bladder did not influence the performance. Although Algorithm B was less affected by the experimental conditions and inter-subject biological variability, it featured higher estimation errors. In vivo validation on suine model showed average errors of 29.36% (volumes greater than 100 ml), demonstrating the potential of the proposed solution and paving the way towards a novel implantable volume monitoring system

Real-time imaging and tracking of microrobots in tissues using ultrasound phase analysis

Ultrasound B-mode imaging has been employed to monitor single agents and collective swarms of microrobots in vitro and ex vivo in controlled experimental conditions. However, low contrast and spatial resolution still limit the effective employment of such a method in a medical microrobotic scenario. Doppler-based ultrasound appears as a promising tool for tracking microrobots in echogenic and dynamic environments as biological tissues. In this Letter, we demonstrate that microrobot displacements can be used as a special signature for their visualization within echogenic media, where B-mode fails. To this aim, we induced vibrations of a magnetic soft microrobot through alternated magnetic fields and used ultrasound phase analysis to derive microrobot features such as size and position over time. By exploiting vibrations, we were able to perform imaging and tracking of a low contrast microrobot both in tissue-mimicking phantom and in chicken breast. The axial resolution was 38 μm, which is four times smaller than the B-mode resolution with the employed equipment. We also performed real-time tracking of the microrobot's positions along linear trajectories with a linear velocity up to 1 mm/s. Overall, the reported results pave the way for the application of the proposed approach for the robust monitoring of medical microrobots in tissue

Soft Robot-Assisted Minimally Invasive Surgery and Interventions: Advances and Outlook

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry's appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery

Contrast-enhanced ultrasound tracking of helical propellers with acoustic phase analysis and comparison with color Doppler

Medical microrobots (MRs) hold the potential to radically transform several interventional procedures. However, to guarantee therapy success when operating in hard-to-reach body districts, a precise and robust imaging strategy is required for monitoring and controlling MRs in real-time. Ultrasound (US) may represent a powerful technology, but MRs' visibility with US needs to be improved, especially when targeting echogenic tissues. In this context, motions of MRs have been exploited to enhance their contrast, e.g., by Doppler imaging. To exploit a more selective contrast-enhancement mechanism, in this study, we analyze in detail the characteristic motions of one of the most widely adopted MR concepts, i.e., the helical propeller, with a particular focus on its interactions with the backscattered US waves. We combine a kinematic analysis of the propeller 3D motion with an US acoustic phase analysis (APA) performed on the raw radio frequency US data in order to improve imaging and tracking in bio-mimicking environments. We validated our US-APA approach in diverse scenarios, aimed at simulating realistic in vivo conditions, and compared the results to those obtained with standard US Doppler. Overall, our technique provided a precise and stable feedback to visualize and track helical propellers in echogenic tissues (chicken breast), tissue-mimicking phantoms with bifurcated lumina, and in the presence of different motion disturbances (e.g., physiological flows and tissue motions), where standard Doppler showed poor performance. Furthermore, the proposed US-APA technique allowed for real-time estimation of MR velocity, where standard Doppler failed

Temperature increase inside LED-based illuminators for in vitro aPDT photodamage studies

Abstract Antimicrobial PhotoDynamic Therapy (aPDT) is an emerging strategy aimed at the eradication of bacterial infections, with a special focus on antibiotic-resistant bacteria. This method is easy to apply, not expensive and particularly interesting in case of bacteria that spontaneously produce the required photosensitizers. In the framework of a project aimed at the development of an ingestible pill for the application of aPDT to gastric infections by Helicobacter pylori, a LED-based illuminating prototype (LED-BIP) was purposely designed in order to evaluate the photodamage induced by light of different wavelengths on porphyrin-producing bacteria. This short paper reports about temperature tests performed to assess the maximum exposure time and light dose that can be administered to bacterial cultures inside LED-BIP without reaching temperatures exceeding the physiological range